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Conditional Expectation (MMSE) 


9.1.4 Conditional Expectation (MMSE) 

Remember that the posterior distribution, fx\Y contains all the knowledge that we 

have about the unknown quantity X . Therefore, to find a point estimate of X, we can just 
choose a summary statistic of the posterior such as its mean, median, or mode. If we choose 
the mode (the value of X that maximizes fx\Y { x \V )X we obtain the MAP estimate of X. 
Another option would be to choose the posterior mean, i.e., 

x = E[X\Y = y\. 

We will show that E\X\Y — y] will give us the best estimate of X in terms of the mean 
squared error. For this reason, the conditional expectation is called the minimum mean 
squared error (MMSE) estimate of X. It is also called the least mean squares (LMS) estimate 
or simply the Bayes' estimate of X. 

Minimum Mean Squared Error ( MMSE ) Estimation 

The minimum mean squared error (MMSE) estimate of the random variable X, given that we 
have observed Y — y, is given by 

x M = E[X\Y = y\. 


Example 9.6 

Let X be a continuous random variable with the following PDF 


fx(x) 


We also know that 


2x if 0 < x < 1 

0 otherwise 


fY\x{y \ x ) 


2 xy — x + 1 if0<y<l 
0 otherwise 


Find the MMSE estimate of X, given Y — y is observed. 

• Solution 

o First we need to find the posterior density, fx\Y (# | V)- We have 
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, , , , fy\x(y\ x )fx(x) 

fx ' Axly) = Mv) 

We can find fy{y) as 

M?/) = [ fr\x(y\x)fx(x)dx 

Jo 

— / (2tn/ — cc + l)2a:dcc 
./o 


r o 

= ^2/+ for 0 < y < 1. 


We conclude 

/xiy(^|y) — 


6x(2xy — x + 1 ) 


4y+ 1 

The MMSE estimate of X given Y — y is then given by 

x M = E[X\Y = y] 


for 0 < x < 1. 


/ xf x \ Y {x\y)dx 
Jo 

If 1 

—— J 6x 2 (2 xy — x + 1 )dx 


4 y 

iy + l 
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